Repository navigation
Add managed llama.cpp lifecycle and settings on macOS arm64 - #60
Merged
Merged
Conversation
Add a bounded tar.gz installer with contained library links, required-file verification, unchanged quarantine attributes and strict existing-signature checks. Stage identity-bound claims with the artifact and atomically exchange owned macOS installations, leaving failed installs and updates unchanged. Preserve sibling model caches on uninstall. This prepares the lifecycle-only port of NVIDIA/Personal-AI-Router@6d2d67f without its proxy plumbing. Co-authored-by: Terve <ntervalon@nvidia.com> Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Adapt the lifecycle-only manifest from NVIDIA/Personal-AI-Router@6d2d67f to the fork, pin b12171 and require its complete server library layout. Protect every audited listener control and port-changing preset, including vendor underscore spellings and inherited environment. Require the /props router identity for readiness and health. Document the unnotarized release, port 8082, retained cache and deferred routing/model integration. Co-authored-by: Terve <ntervalon@nvidia.com> Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Expose llama.cpp installation, runtime state and protected server launch settings without a proxy or model routing. Keep settings revision recovery and collision checks in Go, display the unnotarized build notice, and preserve atomic archive updates from desktop and TUI clients. Adapt the lifecycle-only scope from NVIDIA/Personal-AI-Router@6d2d67f and its follow-ups NVIDIA#153 and NVIDIA#186; the proxy and model hub remain deferred. Reject quarantined source archives without altering their attributes and extend fixture coverage. Co-authored-by: Terve <ntervalon@nvidia.com> Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
The supported archive installer relies on macOS birth-time identities surviving staging-marker writes and directory exchange. Linux intentionally uses change time, so successful directory-claim fixtures cannot describe an install on that unsupported platform. Keep these rollback and cache-preserving uninstall tests in the native Darwin suite while retaining portable extraction, checksum and manifest coverage everywhere. CI exposed both misplaced fixtures; the native suite and cross-platform compilation now pass. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Replace the llama.cpp launch denylist with a closed policy audited against b12171. Validate saved overrides and normalize documented aliases and equals syntax without admitting host tools, remote model sources, filesystem paths or readiness changes. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
JaredP94
force-pushed
the
feat/llamacpp-engine
branch
from
November 6, 2026 20:58
7bbe62b to
3dd35cf
Compare
Launch with only PAIR-owned environment values and child-only HOME/XDG configuration paths. Refuse the unavoidable system config and any config in the controlled user search path before spawn; false reuse-port environment values alone cannot undo a config setting. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
…nd start Review follow-up. Only llama-server's signature was checked, so a damaged adjacent library such as libllama-server-impl.dylib passed, and the same-pin already-installed path skipped the quarantine and signature checks entirely. Every installed Mach-O file, identified by its thin or fat magic bytes rather than name or execute bit, is now checked with codesign --verify --strict --all-architectures (argv, fixed system tool, fail closed). Required files and link containment are revalidated. The checks run on install, on the same-pin return and before every start (under the engine operation lock). Nothing re-signs or clears quarantine. Tests use /usr/lib/dyld, which is always present, signed and universal, as the library fixture. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Review follow-up. The allowlist accepted --mlock, --mmap and --no-mmap, which llama-server b12171 does not define; it rejects unknown arguments, so a saved setting would pass PAIR's validation and then stop the engine from starting. Replace them with the typed --load-mode/-lm option and its six b12171 values. Add a snapshot of b12171's option names and a test that every allowlisted name exists in it, so a stale name fails a test rather than a launch. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
…tions Review follow-up. The snapshot of b12171 option names was extracted from every quoted string starting with a dash, which picked up the value -1 and missed names containing dots such as --fim-qwen-1.5b-default. It now lists exactly the names in each common_arg(...) definition's leading brace groups. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Owner
Author
|
Live-check note before merge (retirement-plan, macOS 27.2 arm64, package built from bfae127): install, start, settings and stop all passed. PAIR installed b12171 with no quarantine, Step f ("nothing routes to llama.cpp") was recorded as incomplete because the Ollama and LM Studio proxies answered Uninstall removed the install directory. No model-cache directory existed, so cache retention on uninstall is covered by unit tests only. The machine was restored to the published 0.9.0 with its data. |
This was referenced Nov 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add llama.cpp as a PAIR-managed engine on macOS arm64: install, start/stop, runtime status, server port, typed inference-only launch arguments and a PAIR-controlled child environment (no user environment assignments). This is PR 1 of 4, adapting selected lifecycle concepts from NVIDIA/Personal-AI-Router@6d2d67f, with upstream author credit on each commit. NVIDIA#153 informs identity-bound installation/removal; NVIDIA#186 remains deferred.
The official b12171 archive is pinned to SHA-256
870ef967970fa18bbd31460bc5f7929bbb5f091359d8dc0f58e20133aa4b5fcd. The macOS build is not notarized, like PAIR itself; the engine lists and documentation say so. It does not serve requests through PAIR yet.Scope
/usr/bin/codesign --verify --strict --all-architectureschecks every Mach-O file, detected by thin/fat magic bytes rather than filename or executable bit: server, adjacent dylibs, extensionless code and all universal slices. Required layout, link containment, quarantine and signature integrity are revalidated on install, the same-pin already-installed path and before every start. Never clear quarantine or re-sign.llamacpp_policy.go, documented aliases inLLAMACPP.md): context, batch and microbatch sizes; threads and batch threads; GPU layers; flash attention; K/V cache types; parallel slots; prediction length; temperature, top-k, top-p and min-p; named built-in chat templates; reasoning format, mode, effort and budget; load mode; continuous batching, KV offload, metrics and slot switches. Numeric ranges and enum choices are typed and bounded. Canonical option names are--ctx-size,--batch-size,--ubatch-size,--threads,--threads-batch,--gpu-layers,--flash-attn,--cache-type-k,--cache-type-v,--parallel,--predict,--temp,--top-k,--top-p,--min-p,--chat-template,--reasoning-format,--reasoning,--reasoning-effort,--reasoning-budget,--load-mode,--cont-batching/--no-cont-batching,--kv-offload/--no-kv-offload,--metrics,--slots/--no-slots.--load-modeacceptsauto/none/mmap/mlock/mmap+mlock/dio; standalone--mlock,--mmapand--no-mmapare rejected. Unknown or repeated options (including aliases/opposing switches), positional arguments,--, non-ASCII, all environment assignments, paths, URLs, model downloads and arbitrary template programs are rejected. Equals syntax is lowered to separate argv values for the pinned parser.LLAMA_ARG_*, cache/download configuration and dynamic-loader overrides cannot leak into the child. PAIR supplies only manifest-owned listener/cache settings, reuse-port disabled, offline mode, fixed system PATH and isolated child HOME/XDG paths. Before every spawn, any file, symlink or inspection failure at/etc/llama.cpp/config.inior{install_dir}/.pair-config/llama.cpp/config.inirefuses launch; the real user configuration is not searched and ambient files are never modified. Invalid saved settings are reported, not silently discarded or applied.TestLlamaAllowlistMatchesPinnedReleasechecks every allowed name/alias againsttestdata/llamacpp-b12171-options.txt, snapshotted from actualcommon_arg(...)option definitions in b12171common/arg.cpp, not incidental option-name mentions. It also covers accepted load modes and rejects stale standalone memory switches. Regenerate the snapshot when changing the release pin.LLAMA_CACHEdirectory. Updating requires stopping the engine first; clients never uninstall it before an update.127.0.0.1:8082, collision/reservation checks,/propsJSONrole=routerreadiness, existing validated LAN-bind path, audited protected listener flags/environment and underscore aliases. Complete audit:services/nvpair-engine-manager/LLAMACPP.md.proxyPort=0), minimal desktop/TUI lifecycle list support and original repository-native icon. No load profiles.Validation
Current-head packaged live check (
bfae1272ab48c84fcdbb91830025bb71fdf4f12b, 2026-11-07; macOS 27.2 arm64, remote Python 3.9.6):/tmp/pair-work/live60worktree usingnpm ci --no-audit --no-fundandnpm run -s build:mac:arm64(both exit 0); no source edits or dependency symlinks.NVPAIR-Setup-0.9.0-arm64.zip: 194752822 bytes, SHA-256e9f737022b9e4798fc8396e651b7f81c3eb9d33d7b5fea09798f41ffd186bacf, verified locally and remotely withshasum -a 256.dittointo~/pair-live60/backup/.diff -rqreturned 0 for both copies (app-data comparison excludedSingleton*andlogs; macOS diff emitted framework-symlink directory-loop warnings). The backup was retained and never modified. Installed the built app and ran only its packaged broker under a Python 3.9-compatible isolated driver over JSON-RPC stdio.engine:get-installed:llamacpp, installed=false,capabilities.install=true.xattr -lrproduced no attributes;codesign --verify --strict --all-architecturesexited 0.spctl --assess --type execute --verbose=4exited 3:llama-server: rejected(not notarized; no bypass attempted)./propsHTTP 200 androle=router;lsofshowed only IPv4127.0.0.1:8082. Environment names only:LLAMA_ARG_API_PREFIX,LLAMA_ARG_HOST,LLAMA_ARG_OFFLINE,LLAMA_ARG_PORT,LLAMA_ARG_REUSE_PORT,LLAMA_CACHE; noHF_*names.--ctx-size 4096preview valid;--host 0.0.0.0,--tools all,-hf org/repo,--mlockeach rejected withoption is not in the llama.cpp inference allowlist. Applied context at revision 2, observed a new healthy PID and literal argv, then restored the exact original settings (serverPort=8082,proxyPort=0, empty launch text) at revision 3 with another healthy restart and/props200/router.lsoffound no listener on 8082./v1/modelson 11434 and 1234 each returned HTTP 503 with exact body{"error":"model inventory unavailable"}; neither supplied a successful inventory, so absence of llama.cpp routing cannot be certified there. Port 1236 returned HTTP 200 with no llama.cpp entries. Stopped further feature testing at the unexpected response and proceeded with cleanup/restoration; this is not evidence that llama.cpp caused those inventory failures.$APP_DATA/engine-bin/llamacpp.$APP_DATA/engine-bin/llamacpp-modelswas absent before and after uninstall; no cache was created to manufacture a retention test.ditto, and verified both restored copies withdiff -rq(exit 0, same exclusions/warnings). Relaunched the published app: version 0.9.0, broker plus all 12 workers running. Deleted~/pair-live60/incoming; kept~/pair-live60/backup/. Removed the temporary local worktree./tmp/pair-review/live-llamacpp/evidence-2.jsonl(JSON lines, including install progress, requests/responses, read-only checks and restoration). No model was downloaded, no other engine was installed/uninstalled, no downloaded archive executable was run directly, and no quarantine/signature/Gatekeeper setting was changed. The earlierhashlib.file_digestharness problem was avoided with Python 3.9-compatible scripts andshasum.Previously recorded validation below is retained as historical evidence; it was not rerun for this live-check attempt and does not certify the current package or complete hosted macOS coverage.
Local macOS arm64, fixture-only; no downloaded release executable run and no real engine installed:
nvpair-engine-manager,nvpair-ui-broker,nvpair-tui):gofmt -l,go vet ./...,GOOS=linux go vet ./...,GOOS=windows go vet ./...,go test -count=1 ./...— all pass (format output empty).desktop:npm ci --no-audit --no-fund;npm run -s typecheck,npm run -s lint(zero warnings),npm run -s test:unit(95 files / 944 tests),npm run -s dead-code:check,npm run -s service-contracts:write,npm run -s service-contracts:check— pass; generated API documentation unchanged.node scripts/spdx-headers.mjs: 0 missing;git diff --check: clean.pgrep -lf 'fake-engine|llama-server': no survivors.TestArchiveContainment,TestArchiveLimitsAndCancellation,TestArchiveChecksumMismatch,TestArchiveQuarantinePreservedAndRejected,TestArchiveQuarantinedSourceRejected,TestArchiveSignatureIntegrity,TestArchiveCodesignUnavailable,TestArchiveInstallRollback,TestArchiveExecutorInstallUpdateRollback,TestArchiveAtomicUpdate,TestArchiveUninstallPreservesModelCacheAndRequiresClaim,TestArchiveManifestValidation,TestLlamaProtectedLaunchInputs,TestLlamaReadinessIdentity,TestLlamaPortCollision, plus broker proxy-free settings/recovery and desktop/TUI lifecycle/update tests. Regression tests observed failing before fixes and passing afterward.7bbe62b3: Desktop checks, SPDX headers, Go modules (ubuntu-latest), and Cross-process tests (ubuntu-latest) all pass. The Go modules job includes the full engine-manager race suite and repeated downloader shutdown regressions.cross-process-ubuntu-latestfrom that recorded run into fresh directory/tmp/pair-llamacpp-final-artifact.6LBRgZ;python3 -I /tmp/pair-review/scripts/noskip.py /tmp/pair-llamacpp-final-artifact.6LBRgZ/cross-process.jsonlreports 123 pass, zero skips/failures.Risk
Other platforms are explicitly unsupported for this engine. Checksum and existing ad-hoc signature integrity do not establish publisher trust or notarization. Installation fails closed on quarantine, a broken signature or unavailable codesign. No quarantine attributes are removed, and no signatures are changed.
Packaged macOS arm64 live install/update/rollback and start/stop smoke checks remain the maintainer's pre-merge task. The current-head attempt stopped at transfer verification before engine testing; no live llama.cpp run is claimed. The two hosted macOS jobs also remain a pre-merge gate because no runner was acquired. Independent automated review could not complete (reviewer timeouts); local source/diff review and the checks above completed. This PR is not an inference or release certification.
Checklist
git commit -s), certifying the Developer Certificate of Origin.services/versions.json, and described user-visible changes above so they reach the release notes.